1. Field of the Invention
The present invention relates generally to the processing of video images, and more particularly to techniques for deinterlacing video images.
2. Description of the Related Art
MPEG-2 makes use of a chroma-to-luma sampling relationship known as 4:2:0, in which two color-difference signals are sampled at one half the rate of luma in both the vertical and horizontal directions. An MPEG-2 decoder typically produces an output signal in the 4:2:2 format in which the color signals are sampled at one half the luma rate in only the horizontal direction. The MPEG decoder must therefore upsample the 4:2:0 encoded data in the vertical direction to produce the desired 4:2:2 output. In addition, MPEG-2 video sequences may be encoded as either a progressive or interlaced image sequence. A problem occurs when an image actually encoded as progressive is decoded as interlaced for the purposes of the chroma upsampling. In this case, the chroma samples are incorrectly upconverted and positioned relative to the corresponding luma samples, resulting in artifacts in areas of vertical color transitions.
MPEG-2 compression is in common use in today's digital video systems. Since MPEG-2 is a digital video compression method, the video signal must be digitally sampled prior to being compressed. The sampled digital video signal is composed of three components: a luminance (aka, ‘luma’) signal which contains brightness information but no color, and two color or chrominance (aka, ‘chroma’) signals which are a scaled version of the difference between red and luma and between blue and luma (aka, ‘R—Y’ or ‘Cr’, and ‘B—Y’ or ‘Cb’, respectively). Inherent in the MPEG-2 encoding is a relationship between the sampling rates and the relative spatial positioning of these luma and chroma samples.
While the MPEG-2 standard supports a variety of signal formats and resolutions, the common type in use today for standard definition video signals uses a luma/chroma sampling relationship known as 4:2:0. This standard dictates that the two chroma signals are sampled at half the rate of the luma signal in both the vertical and horizontal directions—i.e., each of the two chroma signals has one quarter the bandwidth of the luma signal. A chroma sample is spatially positioned halfway between a pair of luma samples in the vertical direction, and may be either halfway between a pair of luma samples in the horizontal direction or may be coincident with one of the two luma samples. It is only the vertical relationship which is at issue here.
Even though MPEG-2 video data is in the 4:2:0 format, most MPEG-2 decoders produce a digital video output which is in the 4:2:2 format. With 4:2:2, the chroma data is sampled at half the luma rate in only the horizontal direction. There is one of each type of chroma sample—Cr and Cb—for each luma sample in the vertical direction. Because of this difference in the input and output sampling rates for chroma, an MPEG-2 decoder must ‘upconvert’ or ‘upsample’ the chroma signals vertically to produce an output in the 4:2:2 format. Various techniques can be used for this upconversion process, and range from simple replication of one chroma sample over two luma samples, to linear interpolation, to more sophisticated sample rate conversion using digital FIR filters.
One other key aspect of MPEG-2 is that the compressed video can be in either progressive or interlaced format. Although the vast majority of MPEG-2 decoders output video in the interlaced format, the original video signal could have been compressed in either of the progressive or interlaced formats, depending on the source's characteristics. Control flags within the MPEG-2 data stream indicate which format was used for each video frame. It is important to note that if a given video signal has been compressed in one format, it must also be decoded in that same format or there can be problems with the positioning of decoded luma and chroma samples relative to each other.
FIG. 1 illustrates what can happen when a video frame of the prior art encoded as progressive is treated as interlaced. The leftmost column 10 indicates a single column of video samples (the horizontal dimension is ignored for the purposes of this discussion) which has been encoded as progressive. For each pair of luma samples 20, represented by the open circles and numbered from zero at the top of the column in increasing order towards the bottom of the column, there is a single chroma sample 30 (note that only one chroma sample 30 per luma pair 20 is shown—there is actually a pair of chroma samples, Cr and Cb) located halfway between the luma pair 20. The chroma samples have been designated with letters, starting with ‘A’ and progressing vertically downwards.
The second column 50 indicates the luma/chroma relationship for images encoded as interlaced. Note that the chroma samples for an interlaced image are not spatially located at equidistant from the luma samples as the chroma samples for a progressive image. The actual chroma samples from the progressive image are derived from an even/odd line pair, not an even/even or odd/odd line pair as for an interlaced image. Even worse, when the chroma samples are vertically upsampled to form the 4:2:2 output, half of the original chroma samples are missing from each of the two fields and the upconversion is performed twice on only half the data. This results in a distortion of the image. This distortion is especially apparent when the interlaced output fields of the MPEG-2 decoder are combined back into the original progressive image by an interlaced-to-progressive video format converter.
The effects of this distortion are shown in the remaining columns in FIG. 1. The third column 60 indicates what takes place when simple sample replication is used to perform the vertical chroma upconversion. The left set of upconverted values 70 are the correct ones based on the original progressive encoding. The right set of upconverted values 80 are those derived from improperly interpreting the video frame as interlaced. Note that the ordering of the ‘interlaced’ chroma upconverted samples actually reverses direction every few samples, resulting in a periodic retrograde sequence. This reversal can result in visible artifacts in the region of vertical chroma transitions such as the border between two objects of different colors. These artifacts somewhat resemble interlace motion artifacts, which is reasonable since they stem from an interlaced scan ordering. The fourth column 90 indicates the same relationship, but assumes that linear interpolation has been used for the upconversion process instead of sample replication. Again, the ordering of the ‘interlaced’ upconverted chroma samples indicates the periodic retrograde sequence which results in visible artifacts.
Accordingly, what is needed is a method and system that detects and repairs video image distortion caused by incorrect decoding of MPEG-2 images.